Skip to content

[AI-179] [Automation] feat(AI-179): implement /fiona search — browse raw sources without synthesized answer - #81

Draft
roberthunterjr with Copilot wants to merge 26 commits into
mainfrom
copilot/ai-179-clone-fiona-search
Draft

[AI-179] [Automation] feat(AI-179): implement /fiona search — browse raw sources without synthesized answer#81
roberthunterjr with Copilot wants to merge 26 commits into
mainfrom
copilot/ai-179-clone-fiona-search

Conversation

Copilot AI commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

/fiona search <query> was a "coming soon" stub. This implements the full skill: calls Perplexity non-streaming, returns 3–5 source snippets with hyperlinks, and deliberately omits any synthesized answer.

Core changes

  • src/agent/llm-caller.js — new searchForSources(query, options): non-streaming Perplexity call that prefers search_results (title + snippet) over citations (URL-only fallback), capped at 5 results
  • src/agent/search-caller.js (new) — abstraction layer re-exporting searchForSources; also owns formatSearchResults() (Slack mrkdwn renderer) and escapeMrkdwn() (HTML-entity-encodes &/</> in user input before embedding in message strings)
  • src/listeners/commands/fiona.jshandleSearch() replaces the stub: empty query falls back to help; rate-limit exceeded responds ephemerally; otherwise ack() immediately → search → respond() with formatted sources; records slash_search
  • src/listeners/commands/command-handler.js — adds handleSearchViaSay() for @-mention/assistant-panel entry points; routeCommandViaSay() dispatches search there; removes SEARCH_NOT_YET_TEXT; drops "(coming soon)" from HELP_TEXT search line

Example output (Slack mrkdwn)

🔍 *Search results for:* _"assessment API endpoints"_

1. *<https://docs.ed-fi.org/assessment|Assessment API — Ed-Fi ODS/API Documentation>*
_"The Assessment API provides endpoints for creating, reading, updating, and deleting assessment metadata…"_

2. *<https://www.ed-fi.org/guide|API Guidelines — Ed-Fi Alliance>*

No results → 🔍 No sources found for _"query"_. Try rephrasing your query.

Test surface

  • tests/agent/search-caller.test.js (new, 38 tests): searchForSources with search_results/citations/empty/error paths; formatSearchResults; escapeMrkdwn
  • tests/listeners/commands/fiona.test.js: search sub-command replaced with real-behavior tests (empty query → help fallback, query → ack+respond+record, missing fields, rate limiting)
  • tests/listeners/commands/command-handler.test.js: mocks search-caller, adds handleSearchViaSay tests
  • Existing app-mention, message, and escalation test files updated: added searchForSources to their llm-caller mocks and updated the "coming soon" search assertions

Copilot AI and others added 3 commits July 23, 2026 16:06
…nthesized answer

- Add searchForSources() to llm-caller.js: non-streaming Perplexity call
  that extracts search_results (with title/snippet) or falls back to
  citations (URL-only), returning up to 5 normalized NormalizedSource items.

- Create src/agent/search-caller.js: thin abstraction layer that re-exports
  searchForSources from llm-caller and provides formatSearchResults() and
  escapeMrkdwn() helpers for Slack mrkdwn formatting.

- Update src/listeners/commands/fiona.js: replace the coming-soon stub for
  /fiona search with a real handleSearch() handler — extracts the query,
  falls back to help when empty, checks rate limits, calls searchForSources,
  and responds ephemerally with formatted source snippets.

- Update src/listeners/commands/command-handler.js: remove SEARCH_NOT_YET_TEXT,
  add handleSearchViaSay() for the @-mention / assistant-panel paths, update
  routeCommandViaSay() to dispatch search to handleSearchViaSay(), and remove
  the (coming soon) annotation from HELP_TEXT.

- Update tests: fiona.test.js, command-handler.test.js, app-mention.test.js,
  message.test.js, escalation.test.js reflect the new real-search behavior.
  New test file: tests/agent/search-caller.test.js (38 unit tests).

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
…brittle test

- Rename handleComingSoonViaSay parameter from subCommand to keyword
- Simplify HELP_TEXT test to use regex instead of exact whitespace match

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
- Move SEARCH_ERROR_TEXT export inline next to its definition in search-caller.js
- Add err.message alongside err.name in handleSearchViaSay error log

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement /fiona search functionality for browsing raw sources feat(AI-179): implement /fiona search — browse raw sources without synthesized answer Jul 23, 2026
Copilot AI requested a review from roberthunterjr July 23, 2026 16:11

try {
const response = await perplexityClient.chat.completions.create({
model: PERPLEXITY_API_MODEL,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is using whatever the default model for pplx is. Currently this is set to sonar, which is the synthesis model. We should update this to use the search api to reduce the adjustments to extracting sources and limiting count, this API will handle most of that. Please look up the guidance on implementing the search API and apply here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched searchForSources to the Perplexity Search API (POST https://api.perplexity.ai/search) via native fetch. The Search API directly returns a structured results array (title, url, snippet) so the search_results/citations extraction and fallback logic has been removed. Count limiting is now delegated to the API via the max_results parameter, with normalizeSources as a secondary cap.

Updated search-caller.test.js to mock globalThis.fetch instead of openai.chat.completions.create, and added a test for HTTP error responses. All 570 tests pass. Commit: feat(AI-179): switch searchForSources to Perplexity Search API (POST /search)

Copilot AI and others added 2 commits July 23, 2026 16:26
…/search)

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>

@roberthunterjr roberthunterjr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot. Looking good overall. A couple of comments from before to address. Also, here is an image of the reponse seen.

Image

}

// ─── Source Search ─────────────────────────────────────────────────────────
const SEARCH_MAX_SOURCES = 5;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was specified in ticket to be 5. Lets make this an env value and set a max gate here to 10, that sends the minimum of the SEARCH_MAX_SOURCES value and 10 when performing the actual API call

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SEARCH_MAX_SOURCES now reads from process.env.SEARCH_MAX_SOURCES (default 5). A hard cap of SEARCH_ABSOLUTE_MAX = 10 is enforced via Math.min(maxSources, 10) before the Perplexity API call — so max_results never exceeds 10 regardless of the env value. Commit: feat(AI-179): env-backed SEARCH_MAX_SOURCES with hard cap

* @param {Array<import('./llm-caller.js').NormalizedSource>} sources - Normalized source list
* @returns {string} Slack mrkdwn-formatted search results string
*/
export function formatSearchResults(query, sources) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking pretty good. One issue I see is the response items appear as a long, unformatted list, making it difficult to read. Lets improve the readabilitiy, perhaps formatting the snippet and truncating the content more so the total number of sources is easier to see.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a truncateSnippet helper in search-caller.js that:

  • Collapses newlines and extra whitespace to a single space
  • Strips **bold** markers (using [\s\S]*? to handle asterisks inside bold text) and ### heading markers
  • Truncates to 150 chars at a word boundary and appends

The SNIPPET_MAX_CHARS limit is also env-backed via SEARCH_SNIPPET_MAX_CHARS (default 150) so operators can tune it. Single-asterisk italic is intentionally not stripped to avoid false positives on math/code notation like a * b. Commit: feat(AI-179): env-backed SEARCH_MAX_SOURCES with hard cap; strip markdown and truncate snippets

Copilot AI and others added 4 commits July 23, 2026 18:37
…down and truncate snippets

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
…ateSnippet

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
…ted to avoid mock breakage

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>

@roberthunterjr roberthunterjr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot A few tweaks here:

Lets use the perplexity client for the search api as seen here, instead of direct API call:
https://docs.perplexity.ai/docs/search/best-practices

Lets ensure the client for Search API also respects the domain filtering defaults or configuration from the .env for filtering.

The snippets revealed as search results are still too long, lets make the snippet truncate after 160 words.

Lastly, lets clear the deliniation between search results. Either lets improve markdown or try something like block kit cards to make the seperation of results and their source and snippets clearer.

Copilot AI and others added 2 commits July 23, 2026 21:58
…er, word truncation, block kit

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
…lock Kit cards

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>

Copilot AI commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

@copilot A few tweaks here:

Lets use the perplexity client for the search api as seen here, instead of direct API call:...

Done in commit feat(AI-179): use Perplexity SDK, domain filter, 160-word snippets, Block Kit cards.

  • Perplexity SDKsearchForSources now uses @perplexity-ai/perplexity_ai (client.search.create(...)) instead of raw fetch; the package is added as a dependency.
  • Domain filteringsearch_domain_filter: PERPLEXITY_DOMAIN_FILTER is passed to every search.create call, so the PERPLEXITY_DOMAIN_FILTER env var (defaulting to www.ed-fi.org,docs.ed-fi.org) applies to search the same way it does to chat completions.
  • 160-word truncationtruncateSnippet now counts words instead of characters, capped at 160 (configurable via SEARCH_SNIPPET_MAX_WORDS). Both env vars are documented in .env.sample.
  • Block Kit cardsformatSearchResults now returns { text, blocks }. Each result is a section block (numbered title link) + context block (snippet), separated by divider blocks for clear visual separation. The plain-text text field serves as a notification/accessibility fallback. All callers updated to spread { text, blocks } into respond()/say().

@roberthunterjr roberthunterjr changed the title feat(AI-179): implement /fiona search — browse raw sources without synthesized answer [AI-179] [ Jira Automation] feat(AI-179): implement /fiona search — browse raw sources without synthesized answer Jul 28, 2026
@roberthunterjr roberthunterjr changed the title [AI-179] [ Jira Automation] feat(AI-179): implement /fiona search — browse raw sources without synthesized answer [AI-179] [Automation] feat(AI-179): implement /fiona search — browse raw sources without synthesized answer Jul 28, 2026
Copilot AI review requested due to automatic review settings July 30, 2026 19:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements the /fiona search <query> capability in the Fiona Slack app by calling Perplexity’s Search API (non-streaming) and returning 3–5 raw source snippets/links without any synthesized answer. The PR also expands the feedback/telemetry plumbing to tag feedback with response/interaction context (e.g., synthesis vs search; slash vs mention) and updates the command routing so search works via both slash commands and say()-based entry points.

Changes:

  • Add a search abstraction (search-caller) that re-exports the LLM-layer search call and formats results as Slack mrkdwn + Block Kit (with snippet truncation + escaping).
  • Implement /fiona search and say()-based “search” command handling, including rate limiting, immediate ack(), formatted responses, and slash_search interaction recording.
  • Extend feedback blocks/handling to encode response type + interaction type in block_id, and record those fields in Cosmos feedback records; update tests accordingly and add the Perplexity Search SDK dependency.

Reviewed changes

Copilot reviewed 24 out of 25 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
apps/fiona-slack/src/agent/llm-caller.js Adds Perplexity Search API client + searchForSources() implementation.
apps/fiona-slack/src/agent/search-caller.js New wrapper for search + Slack formatting/escaping/snippet truncation.
apps/fiona-slack/src/listeners/commands/fiona.js Implements /fiona search flow (ack → search → respond), rate limiting, and feedback block tagging.
apps/fiona-slack/src/listeners/commands/command-handler.js Adds say()-based search handler + routes “search” keyword to it; updates help text.
apps/fiona-slack/src/listeners/commands/command-dispatch.js Threads interactionType through keyword routing for feedback tagging.
apps/fiona-slack/src/listeners/actions/feedback.js Parses contextual feedback block ids; attempts to extract search query for search feedback.
apps/fiona-slack/src/listeners/views/feedback_block.js Adds response/interaction-aware feedback block ids + helpers; exports FEEDBACK_RESPONSE_TYPES.
apps/fiona-slack/src/listeners/views/feedback_reason.js Records feedback with response/interaction context; fetches context differently for synthesis vs other response types.
apps/fiona-slack/src/listeners/events/app_mention.js Tags synthesis feedback blocks with interaction type for mentions.
apps/fiona-slack/src/listeners/assistant/message.js Tags synthesis feedback blocks with interaction type for assistant-thread messages.
apps/fiona-slack/src/agent/feedback-store.js Records responseType in Cosmos feedback documents (defaults to synthesis).
apps/fiona-slack/src/agent/feedback-response-types.js New shared enum for feedback response categories.
apps/fiona-slack/tests/agent/search-caller.test.js New test suite for search normalization/formatting/escaping and error paths.
apps/fiona-slack/tests/listeners/commands/fiona.test.js Replaces search stub tests with real search behavior + rate limit coverage.
apps/fiona-slack/tests/listeners/commands/command-handler.test.js Adds tests for say()-based search routing/formatting and help text updates.
apps/fiona-slack/tests/listeners/events/app-mention.test.js Updates mention “search …” expectations from coming-soon to search results.
apps/fiona-slack/tests/listeners/assistant/message.test.js Updates assistant-thread “search …” expectations from coming-soon to search results.
apps/fiona-slack/tests/listeners/actions/feedback.test.js Updates feedback action tests to include contextual block_id and metadata.
apps/fiona-slack/tests/listeners/views/feedback-block.test.js Adds coverage for feedback block id build/parse + custom feedback blocks.
apps/fiona-slack/tests/listeners/views/feedback_reason.test.js Adds search-feedback recording tests and conversation.history mocking.
apps/fiona-slack/tests/agent/feedback-store-cosmos.test.js Adds responseType field coverage + defaulting behavior.
apps/fiona-slack/tests/agent/escalation.test.js Updates llm-caller mock to include searchForSources.
apps/fiona-slack/package.json Adds @perplexity-ai/perplexity_ai dependency.
apps/fiona-slack/package-lock.json Locks @perplexity-ai/perplexity_ai@0.37.0.
apps/fiona-slack/.env.sample Documents optional /fiona search env settings.
Files not reviewed (1)
  • apps/fiona-slack/package-lock.json: Generated file
Comments suppressed due to low confidence (3)

apps/fiona-slack/src/listeners/actions/feedback.js:36

  • extractSearchQuery() returns the mrkdwn-escaped query (e.g., &amp;, &lt;, &gt;), which then gets stored as userMessage for search feedback. Unescape these entities so feedback analytics/store reflect the user’s original input.
function extractSearchQuery(messageText) {
  if (typeof messageText !== 'string') return null;
  const match = messageText.match(SEARCH_QUERY_PATTERN);
  return match?.[1] ?? null;
}

apps/fiona-slack/src/listeners/actions/feedback.js:95

  • To avoid relying on conversations.history() (which won’t return ephemeral messages), include the rated message text in private_metadata when opening the feedback modal. This lets the view submission callback record botResponse even for ephemeral /fiona search responses.
        private_metadata: JSON.stringify({
          channelId: channel_id,
          messageTs: message_ts,
          userId: user_id,
          value,

apps/fiona-slack/src/listeners/views/feedback_reason.js:91

  • For non-synthesis feedback you fetch the rated message via conversations.history(). That won’t work for ephemeral messages (e.g., /fiona search responses), so botResponse will frequently be null in stored feedback. Prefer passing body.message.text through the modal private_metadata in feedbackActionCallback and using it as the primary (or fallback) botResponse here.
      if (normalizedResponseType === FEEDBACK_RESPONSE_TYPES.SYNTHESIS) {
        ({ userMessage, botResponse } = await fetchThreadContext(client, channelId, thread_ts, messageTs));
      } else {
        botResponse = await fetchMessageText(client, channelId, messageTs);
      }

const { channelId, messageTs, userId, value, thread_ts, responseType, interactionType, searchQuery } = JSON.parse(
view.private_metadata,
);
const normalizedResponseType = responseType ?? FEEDBACK_RESPONSE_TYPES.SYNTHESIS;

import { FEEDBACK_RESPONSE_TYPES, parseFeedbackBlockId } from '../views/feedback_block.js';

const SEARCH_QUERY_PATTERN = /^🔍 \*Search results for:\* _"([\s\S]+?)"_/;
if (!perplexitySearchClient) return [];
if (!query || !query.trim()) return [];

const cappedMaxSources = Math.min(maxSources, SEARCH_ABSOLUTE_MAX);
Comment on lines +41 to +47
// Collapse newlines and runs of whitespace to a single space
let cleaned = text
.replace(/\n+/g, ' ')
.replace(/\s{2,}/g, ' ')
.trim();
// Strip bold markers (**text**) and heading markers (## …)
cleaned = cleaned.replace(/\*\*([\s\S]*?)\*\*/g, '$1').replace(/#{1,6}\s+/g, '');
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
@roberthunterjr

Copy link
Copy Markdown
Contributor

@copilot Please perform another code review that includes the new changes

Copilot AI and others added 2 commits July 31, 2026 21:05
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>

Copilot AI commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Please perform another code review that includes the new changes

Ran another code review, fixed the follow-up issues it surfaced, and re-checked the updated changes. Latest fix is in 4aa6416; npm run lint and npm test both pass, and the final review found no significant issues.

@roberthunterjr

Copy link
Copy Markdown
Contributor

@copilot This is looking nice. I notice that when performing the slash search with an at mention, the results are posted in the channel directly rather than ephemerally. Lets correct this to adhere to the acceptance criteria of the ticket.

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>

Copilot AI commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@copilot This is looking nice. I notice that when performing the slash search with an at mention, the results are posted in the channel directly rather than ephemerally. Lets correct this to adhere to...

Addressed in 005d3d7. @fiona search ... from an app mention now uses chat.postEphemeral instead of say(), so the results stay visible only to the invoking user. I also added a regression test for the app-mention search path; npm run lint and npm test both pass.

@roberthunterjr

Copy link
Copy Markdown
Contributor

@copilot The search is working in the side panel and in the Fiona DM, however tagging in a channel with the search keyword does not yield any results, neither ephemeral or posted as a thread directly.

Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>

Copilot AI commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@copilot The search is working in the side panel and in the Fiona DM, however tagging in a channel with the search keyword does not yield any results, neither ephemeral or posted ...

Fixed in 098186a. Top-level channel @fiona search ... no longer sends the ephemeral result into an implicit thread, so the response is visible in-channel to the invoking user again; threaded mentions still keep the ephemeral response in-thread. Added regression coverage for both cases, and npm run lint plus npm test both pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants